1. Field of the Invention
The invention concerns testing techniques generally and more particularly concerns automated techniques for testing mastery of a body of knowledge.
2. Description of Related Art
One of the most efficient ways of finding out whether someone has mastered a body of knowledge is to give the person a test made up of questions about the body of knowledge and seeing how well the person has answered the questions. In a society which values opportunity and fairness on the one hand and requires technical competence on the other, testing has become enormously important. Test results determine not only an individual's access to educational opportunities or to trades and professions, but are also used to determine the quality of the institutions where the individuals receive their training.
One consequence of the ubiquity and importance of tests in our society has been the development of techniques for validating the questions used in the tests. What is at issue in validation is how well the question discriminates those who have mastered the subject matter in question from those who have not. One technique for validation is dividing the test takers into three groups according to the number of right answers they gave in their tests. The first group contains the test takers that did best; the second group contains the test takers that did worst; the third group contains the average test takers. A given question is validated by comparing the percentage of takers in the first group that answered the question correctly with the percentage of takers in the second group that answered it correctly; the larger the percentage of the first group relative to the second group, the better the question discriminates. See Truman J. Kelley, "The Selection of Upper and Lower Groups for the Validation of Test Items", Journal of Educational Psychology, Vol. 30, 1939, 17-24.
Another consequence has been the development of "objective" testing, that is, testing where the test grader's subjective opinion of the student or of the work he is correcting has no influence on the grade. Examples of objective tests are true-false tests or multiple-choice tests. Objective tests have two advantages: first, they ensure fairness in test grading if not in test making; second, they permit automation of the whole process of taking tests, grading tests, and evaluating tests. The automation possible with objective tests has made it possible to administer tests such as the Scholastic Aptitude Test, the Law School Admissions Test, or the Multi-state Bar Examination to enormous numbers of people; it has also lead to the development of a large industry devoted to developing, validating, giving, and grading automated tests.
While objective tests have traditionally been given and graded in batch mode, that is, a group of test takers takes the test using a machine-gradable answer sheet, the sheets are collected, the machine grades them, and the result is returned to the students, the enormous reduction in the cost of computers has also made it possible to give objective tests interactively: the student sits at a terminal, receives each question and the possible answers in turn, and selects a response. The computer to which the terminal is connected can immediately respond to the answer; for example, the computer may terminate the test early if the taker has already done enough to demonstrate his or her competence or incompetence or may determine from the results thus far what areas require further testing and present more questions in those areas.
Automated objective tests have certainly increased the amount and fairness of access to opportunity. Automation has made it possible for enormous numbers of people to take the tests and has also made it possible for them to take the tests literally anywhere. Further, the results of properly-made objective tests have turned out to be good indicators of the taker's mastery of a body of knowledge and of the taker's probable success in applying the knowledge. It has thus become possible for an institution of higher learning, for example, to make a rational judgment about admitting a student about whom the institution of higher learning knows nothing beyond the test scores.
While the success of objective testing is undeniable, objective tests as heretofore made and used have a number of problems. At the root of these problems is the fact that a student's ability to answer in an objective test is necessarily limited to choosing among the responses offered by the test writer. There is no opportunity for the student to actively demonstrate what he or she can do with the subject matter. One consequence of this fact is that taking objective tests is much less interesting than taking other kinds of tests. Another is that students often spend more time and effort figuring out how to "work" an objective test than they do learning the subject matter. Another, more serious consequence, is that if there is a mismatch between what the student knows and what the test asks, there is no way for the student to get outside the questions provided him or her and show what he or she can really do. It is an object of the invention disclosed herein to overcome these and other problems of objecting testing with techniques that make objective tests tools for learning as well as for finding out what the test takers know.